Assessing the Performance of Artificial Intelligence Models: Insights from the American Society of Functional Neuroradiology Artificial Intelligence Competition.

BACKGROUND AND PURPOSE: Artificial intelligence (AI) models in radiology are frequently developed and validated using datasets from a single institution and are rarely tested on independent, external datasets, raising questions about their generalizability and applicability in clinical practice. The American Society of Functional Neuroradiology (ASFNR) organized a multi-center AI competition to evaluate the proficiency of developed models in identifying various pathologies on NCCT, assessing age-based normality and estimating medical urgency. MATERIALS AND METHODS: In total, 1201 anonymized, full-head NCCT clinical scans from five institutions were pooled to form the dataset. The dataset encompassed normal studies as well as pathologies including acute ischemic stroke, intracranial hemorrhage, traumatic brain injury, and mass effect (detection of these-task 1). NCCTs were also assessed to determine if findings were consistent with expected brain changes for the patient's age (task 2: age-based normality assessment) and to identify any abnormalities requiring immediate medical attention (task 3: evaluation of findings for urgent intervention). Five neuroradiologists labeled each NCCT, with consensus interpretations serving as the ground truth. The competition was announced online, inviting academic institutions and companies. Independent central analysis assessed each model's performance. Accuracy, sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curves were generated for each AI model, along with the area under the ROC curve (AUROC). RESULTS: 1177 studies were processed by four teams. The median age of patients was 62, with an interquartile range of 33. 19 teams from various academic institutions registered for the competition. Of these, four teams submitted their final results. No commercial entities participated in the competition. For task 1, AUROCs ranged from 0.49 to 0.59. For task 2, two teams completed the task with AUROC values of 0.57 and 0.52. For task 3, teams had little to no agreement with the ground truth. CONCLUSIONS: To assess the performance of AI models in real-world clinical scenarios, we analyzed their performance in the ASFNR AI Competition. The first ASFNR Competition underscored the gap between expectation and reality; the models largely fell short in their assessments. As the integration of AI tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings.ABBREVIATIONS: AI = artificial intelligence; ASFNR = American Society of Functional Neuroradiology; AUROC = area under the receiver operating characteristic curve; DICOM = Digital Imaging and Communications in Medicine; GEE = generalized estimation equation; IQR = interquartile range; NPV = negative predictive value; PPV = positive predictive value; ROC = receiver operating characteristic; TBI = traumatic brain injury.

Adenoid cystic carcinoma: A case of rare breast cancer.

Adenoid cystic carcinoma is a rare form of breast cancer accounting for 0.1%-1.0% of all mammary malignancies. It is characterized by an indolent clinical course and favorable prognosis, contrary to other breast cancers. Diagnostic mammogram and breast ultrasound play a pivotal role in the early detection and diagnosis of breast adenoid cystic carcinoma. Treatment may consist of lumpectomy and radiation therapy vs mastectomy alone. Even though rare, late disease recurrence and metastasis has been reported in the literature thus long-term surveillance is of utmost importance for these patients. We will review the literature and discuss the case of a 52-year-old female who presented with a palpable lump of the right breast, which was pathologically proven to be adenoid cystic carcinoma of the breast.

Vertebral Artery Dissection: A Pain in the Neck.

Vertebral artery dissection (VAD) is increasingly identified as a cause of ischemic stroke in young adults. Patients most commonly present with neck pain, headache, visual disturbance, or focal extremity weakness. We present a case of spontaneous VAD in a patient whose only symptoms at presentation were neck pain and headache. A 42-year-old male presented to the emergency department with one week of left neck pain and headache. Computed tomography (CT) neck with contrast was initially ordered for neck pain. CT neck revealed an incidental anterior communicating artery (ACOM) aneurysm. Digital subtraction angiography (DSA) performed for ACOM aneurysm coiling demonstrated a left VAD, which was the attributable etiology to the patient's presentation. Subsequent magnetic resonance angiogram (MRA) neck confirmed this finding. Follow-up brain MRI revealed a small acute left occipital lobe infarct secondary to thromboembolism from the VAD. The patient underwent endovascular coiling of the ACOM aneurysm and received aspirin for the VAD, obtaining resolution of his symptoms. VAD involves an intimal tear of the vasa vasorum leading to narrowing of the vessel lumen that can result in thromboembolic complications. Risk factors for development of VAD include neck manipulations, trauma, or abnormal posturing. DSA remains the gold standard imaging exam for diagnosis of VAD. However, recognition of VAD on more common non-invasive modalities, such as computed tomography angiogram or MRA, remains critical for establishing the correct diagnosis. Although the clinical presentation of VAD is highly variable, dissection should be considered in a young patient with craniocervical pain, even in the absence of neurological symptoms. Early diagnosis and treatment of VAD can lower the risk of long-term neurologic sequelae.

Rapid-sequence brain magnetic resonance imaging for Chiari I abnormality.

OBJECTIVE Fast magnetic resonance imaging (fsMRI) sequences are single-shot spin echo images with fast acquisition times that have replaced CT scans for many conditions. Introduced as a means of evaluating children with hydrocephalus and macrocephaly, these sequences reduce the need for anesthesia and can be more cost-effective, especially for children who require multiple surveillance scans. However, the role of fsMRI has yet to be investigated in evaluating the posterior fossa in patients with Chiari I abnormality (CM-I). The goal of this study was to examine the diagnostic performance of fsMRI in evaluating the cerebellar tonsils in comparison to conventional MRI. METHODS The authors performed a retrospective analysis of 18 pediatric patients with a confirmed diagnosis of CM-I based on gold-standard conventional brain MRI and 30 controls without CM-I who had presented with various neurosurgical conditions. The CM-I patients were included if fsMRI studies had been obtained within 1 year of conventional MRI with no surgical intervention between the studies. Two neuroradiologists reviewed the studies in a blinded fashion to determine the diagnostic performance of fsMRI in detecting CM-I. For the CM-I cohort, the fsMRI and T2-weighted MRI exams were randomized, and the blinded reviewers performed tonsillar measurements on both scans. RESULTS The mean age of the CM-I cohort was 7.39 years, and 50% of these subjects were male. The mean time interval between fsMRI and conventional T2-weighted MRI was 97.8 days. Forty-four percent of the subjects had undergone imaging after posterior fossa decompression. The sensitivity and specificity of fsMRI in detecting CM-I was 100% (95% CI 71.51%-100%) and 92.11% (95% CI 78.62%-98.34%), respectively. If only preoperative patients are considered, both sensitivity and specificity increase to 100%. The authors also performed a cost analysis and determined that fsMRI was significantly cost-effective compared to T2-weighted MRI or CT. CONCLUSIONS Despite known limitations, fsMRI may serve as a useful diagnostic and surveillance tool for CM-I. It is more cost-effective than full conventional brain MRI and decreases the need for sedation in young children.

Uncommon Implantation Sites of Ectopic Pregnancy: Thinking beyond the Complex Adnexal Mass.

Ectopic pregnancy occurs when implantation of the blastocyst takes place in a site other than the endometrium of the uterine cavity. Uncommon implantation sites of ectopic pregnancy include the cervix, interstitial segment of the fallopian tube, scar from a prior cesarean delivery, uterine myometrium, ovary, and peritoneal cavity. Heterotopic and twin ectopic pregnancies are other rare manifestations. Ultrasonography (US) plays a central role in diagnosis of uncommon ectopic pregnancies. US features of an interstitial ectopic pregnancy include an echogenic interstitial line and abnormal bulging of the myometrial contour. A gestational sac that is located below the internal os of the cervix and that contains an embryo with a fetal heartbeat is indicative of a cervical ectopic pregnancy. In a cesarean scar ectopic pregnancy, the gestational sac is implanted in the anterior lower uterine segment at the site of the cesarean scar, with thinning of the myometrium seen anterior to the gestational sac. An intramural gestational sac implants in the uterine myometrium, separate from the uterine cavity and fallopian tubes. In an ovarian ectopic pregnancy, a gestational sac with a thick hyperechoic circumferential rim is located in or on the ovarian parenchyma. An intraperitoneal gestational sac is present in an abdominal ectopic pregnancy. Intra- and extrauterine gestational sacs are seen in a heterotopic pregnancy. Two adnexal heartbeats suggest a live twin ectopic pregnancy. Recognition of the specific US features will help radiologists diagnose these uncommon types of ectopic pregnancy.